The shadow regions in an image may lead to uncertainty of the image content, which is not conducive to other computer vision tasks, so shadow detection is often considered as a pre-processing process of computer vision algorithms. However, most of the existing shadow detection algorithms use a multi-level network structure, which leads to difficulties in model training, and although some algorithms adopting single-layer network structure have been proposed, they only focus on local shadows and ignore the relation between shadows. To solve this problem, a shadow detection algorithm based on hybrid attention model was proposed to improve the accuracy and robustness of shadow detection. Firstly, the pre-trained deep network ResNext101 was used as the front-end feature extraction network to extract the basic features of the image. Secondly, the bidirectional pyramid structure was used for feature fusion from shallow to deep and deep to shallow, and an information compensation mechanism was proposed to reduce the loss of deep semantic information. Thirdly, a hybrid attention model was proposed for feature fusion by combining spatial attention and channel attention, so as to capture differences between shaded and non-shaded regions. Finally, the prediction results of two directions were merged to obtain the final shadow detection result. Comparison experiments were conducted on public datasets SBU and UCF. The results show that compared with DSC (Direction-aware Spatial Context) algorithm, the Balance Error Rate (BER) of the proposed algorithm is reduced by 30% and 11% respectively, proving that the proposed method can better suppress shadow error detection and enhance shadow details.